Avoiding Communication in Primal and Dual Block Coordinate Descent Methods

Avoiding communication in primal and dual block coordinate descent methods

Primal and dual block coordinate descent methods are iterative methods for solving regularized and unregularized optimization problems. Distributed-memory parallel implementations of these methods have become popular in analyzing large machine learning datasets. However, existing implementations communicate at every iteration which, on modern data center and supercomputing architectures, often ...

Randomized Primal-Dual Proximal Block Coordinate Updates

In this paper we propose a randomized primal-dual proximal block coordinate updatingframework for a general multi-block convex optimization model with coupled objective functionand linear constraints. Assuming mere convexity, we establish its O(1/t) convergence rate interms of the objective value and feasibility measure. The framework includes several existingalgorithms as s...

Sparse Linear Programming via Primal and Dual Augmented Coordinate Descent

Over the past decades, Linear Programming (LP) has been widely used in differentareas and considered as one of the mature technologies in numerical optimization.However, the complexity offered by state-of-the-art algorithms (i.e. interior-pointmethod and primal, dual simplex methods) is still unsatisfactory for problems inmachine learning with huge number of variables and constr...

Smooth Primal-Dual Coordinate Descent Algorithms for Nonsmooth Convex Optimization

We propose a new randomized coordinate descent method for a convex optimization template with broad applications. Our analysis relies on a novel combination of four ideas applied to the primal-dual gap function: smoothing, acceleration, homotopy, and coordinate descent with non-uniform sampling. As a result, our method features the first convergence rate guarantees among the coordinate descent ...

Block coordinate descent methods for semidefinite programming